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ABSTRACT 

Information Based Evaluation (IEE) is identified as a 
design procedure for assessing a variety of projects, programs, and 
educational changes. IBE vas used to evaluate a Comprehensive 
Diagnostic Teaching Center (DTC) vhich, in addition to providing 
teacher training and services to handicapped pupils, vpuld bring 
together and focus all of the resources of the Indianapolis Public. 
Schools on this problem. Students vere referred to the DTC by the 
school psychologist usually in cooperation vith the classroom 
teacher. The philosophical approach vhich guided the diagnostic 
activities of the DTC is that of diagnostic/prescriptive teaching 
seen as an on going educational process v herein instructional 
strategies, selected as a result of diagnostic activities are matched 
to a student 9 s unique learning mode. IBE places the information needs 
of concerned persons as the overriding concern of any evaluation. 
Reasons are stated indicating the inadequacy of traditional 
performance objective evaluation. Hovever, IBE is not ft.ee of the use 
of objectives; their use is justified when feedback is important to 
information users, When looking at the evaluation process for any 
program, it is recommended that one concept of evaluation is adopted 
and maintained through the life of the project insofar as possible. 
(RC) 
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This presentation will be concerned with the use of Information 
Based Evaluation (IBE) by a consulting firm to evaluate the Diagnostic 
Teaching Center (DTC) of the Indianapolis Public Schools and should 
be viewed within the perspective of this symposium. It will not present 
tables of chi square, correlation co-efficients, nor even mention multiple 
regression analysis. Thus, while it might appear to be statistically bor- 
ing, it might also be, for an AERA convention, curiously refreshing. i 

For a complete perspective of the relationship that IBE has had with 
the DTC, it will be necessary to give a historical account of the develop- 
ment of the DTC in some detail. This information will allow you to formu- 
late evaluative questions that are of interest to you regarding the project. 
Following the information on the development and operational procedures 
of the DTC, evaluation activities that have been employed throughout the 
life of the DTC will be discussed. 

Development of the DTC 

The need for a diagnostic teaching center was crystallized by a survey 
taken in the light of legislation passed by the 1969 legislature of Indiaha. 
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This act mandated each school corporation in Indiana to provide educational 
programs for all handicapped pupils between the ages of six and 18 years of 
age by 1973. The mandate is now operational. The survey was conducted 
with parents of handicapped and non-handicapped children, with students, 
and with personnel in community agencies such as child guidance clinics 
and the Indiana University medical school. Reports within the public schools 
from psychologists, classroom teachers of exceptional children, social 
service and guidance personnel, and elementary school principals were 
indicative that such a need for a comprehensive center did exist. 

By looking at staff availability, the number of children who would 
need service, and the number of persons presently employed or revealed 
as needed by the survey, it was apparent that utilizing the present approach 
to meeting the needs of the handicapped children while attempting to comply 
with the mandated legislation would be both difficult and expensive to effect. 

Thus, there was determined a need to improve the quality and/or 
the direction of the present activities of the classroom teachers so they 
might attain greater competency in providing for the borderline handicapped 
child who has the potential for succeeding in the regular classroom with a 
specialized program. 

In addition, there was the clearly defined need for a facility which, 
in addition to providing teacher training and services to handicapped pupils, 
would bring together and focus all the resources of the Indianapolis Public 
Schools on this problem. (Proposal for a Comprehensive Diagnostic Teaching 
Center, 1971). 
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The proposal for federal funding of a Diagnostic Teaching Center 
was written by the director of special education and the director of psy- 
chological services of the Indianapolis Public Schools. It was submitted, 
approved, and became funded in July of 1971 . 

It was decided to house the DTC in a regular elementary school 
building. A ten-year-old school centrally located within the boundaries 
of the school district was chosen. The school building had terminated its 
program for junior high students, and the seven rooms left vacant were 
allocated to the DTC . 

A project director was hired in August and his initial task was to 
secure staff. In this system, all staff are hired via a screening committee 
of three or four people. AH policies, rules, and regularions of the system 
also pertain to the operation of the DTC. Seven teachers with regular 
elementary teaching experience were employed along with two teacher 
aides, a psychologist, a reading specialist, and two secretaries. 

After 13 weeks of inservice training, the teachers and staff were 
ready to begin their diagnostic/prescriptive work with students. Several 
plans were tried in order to determine an appropriate case load which 
would allow the development of a quality product containing recommendations 
for the regular classroom teacher to use. 

The management training which was emphasized by the evaluator 
in the early days of the DTC proved valuable in that the project director 
established a system for constant feedback from most activities. When the 
feedback indicated that a function was not operating in such a manner to 
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assist in goal attainment, the function was changed. 

For instance, the teachers 1 response to a feedback evaluation form 
in the first year resulted in changing the content of the teacher training sessions. 
The module on perceptual-motor development was not felt to be useful and 
the teachers were hesitant to assume remedial activities in this area. The 
module, per se, was dropped and the educational prescriptions were aUored 
to be more practical. 

Feedback from the staff personnel when they conducted their follow- 
up activities suggested that the time factor alone made appropriate follow-up 
increasingly less likely as more students were processed through the DTC . 
Consequently, for the second year, follow-up specialists were employed. 

During the second year, the follow-up people indicated that the class- 
room teachers had not become as familiar with the D/P Guide as was felt 
necessary to adequately implement the recommendations. Consequently, the 
teacher training component was changed to have time for reading of the Guide 
during the inservice session. 

Many changes were made during the first two years of the project 

several as a direct result of analyzing feedback information from a variety 
of sources and of having the administrative freedom to act positively on that 
feedback information. By now, the operational procedures are well established 
and a description of them follows. 

Students are referred to the DTC by the school psychologist usually 
in cooperation with the classroom teacher. The student must display an 
educationally significant d iscrepancy between his potential and his academic 
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achievement. How this is done is left up to the professional decision 
of the classroom teacher and the school psychologist. One common 
method Is to compare WISC to scores with standard scores of the 
WRAT, being aware of certain pitfalls when doing so* The school 
psychologist obtains parent permission before the referral to the 
DTC is made. 

Accepted students are transferred to the DTC and transported 
via the regularly established special education bus routes. Sixteen 
students are enrolled at the DTC in phases. Each phase lasts four 
weeks. Four teachers are responsible for conducting complete 
diagnostic/prescriptive activities with four students. One teacher 
provides an instructional program for all 16 students in a classroom 
setting with the assistance of a teachers aide. The students are 
removed from their classroom individually and in small groups for 
diagnostic activities and for trial teaching. This year two staff 
members were assigned to two regular elementary school buildings 
where they perform similar duties with teacher referred students 
instead of at the DTC giving service much quicker. 

The diagnostic/prescriptive (D/P) teachers spend the first 
two weeks of each phase in rather intensive testing of her four stu- 
dents. A minimal test battery given to each student consists of: 

Informal Reading Inventory (Sucher-Allred) 
Informal Math Inventory (Math Modules) 
IT PA 
We p man 
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Bender 
F^stig 
PPMS 
Slingerland 

These data are recorded on a Staffing Review Form and are 
discussed in staffings for each student during the third week of each 
phase. Results of the staffings are taken into consideration during 
trial teaching for the remainder of the third and fourth weeks. In 
addition to the diagnostic work, the D/P teachers contact the parents 
to obtain pertinent medical and background information. Also, they 
write summaries of the diagnostic findings and compile from a bank 
of prescriptions, educational prescriptions appropriate for each stu- 
dent based on diagnostic findings. If no prescription exists, one is 
written and located in the file according to the retrieval plan. 

Diagnostic activities are divided into Cognitive, Affective, 
and Psychomotor areas. The prescription file and retrieval system 
is organized to match the Analysis Sheets on which the diagnostic 
information is recorded. The academic areas of Language, Reading, 
and Math are filed under the Cognitive domain. 

The project employes a consulting optometrist who provides 
a visual screening examination for all students and a consulting 
neurologist who evaluates those students of concern to the staff. 
The Language Specialist conducts hearing testing, assists in the 
administration and the scoring of the Slingerland, provides language 
development activities, and writes prescriptions for the classroom 



teacher. The Reading Specialist assists the D/P teachers in reading 
diagnosis, writes appropriate prescriptions, and conducts the teacher 
training sessions on Reading and materials development. 

The philosophical approach which guides the diagnostic 
activities of the DTC is that of diagnostic/prescriptive teaching seen 
as an ongoing educational process wherein instructional strategies, 
selected as a result of diagnostic activities are matched to a student's 
unique learning mode. The definition of each student's learning mode 
then is the crux of the diagnostic process. The Learning Mode involves 
the determination of the appropriate task level of instruction, the student' 
preferred sensory modality relative to how he processes information, 
and instructional situations which insure learning. When appropriate 
instructional strategies are matched to this determined learning mode, 
then learning is more likely to occur. 

During the last week of each phase each D/P teacher builds a 
D/P Guide, a booklet containing the summary of test findings in both 
raw data form and in narrative description in the following six areas: 
Learning Mode, Language, Reading, Math, Affective, and Psychor- 
motor. Following each summary, a few objectives are listed for the 
student to attain within a relatively short period of time. These 
objectives are used as checkpoints by the Learning Specialists during 
their follow-up activities. One copy of the D/P guide goes to the receiv- 
ing classroom teacher and one copy remains on file at the DTC for 
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reference and use by the Learning Specialists, The task of creating 
the D/P Guides depends on a coordinated effort by all staff so that 
information is available to the clerks in time for the typing to be 
completed in time for the pages to be assembled, punched and bound 
in time for the teachers to read and review during the specified 
teacher trainig session. 

For each phase, a different D/P teacher serves as class 
teacher for the instruction of the 16 students at the DTC. This plan 
allows each D/P teacher to remember what is involved in actual 
instructing a classroom of students* By keeping in touch, the recom- 
mendations made in the D/P Guide are meaningful and practical. 

An important component of the project is training of the 
teachers who will be receiving the students from their 4-week stay 
at the DTC. The receiving classroom teachers come to the DTC on 
Tuesdays for 3 weeks. The project provides substitutes for these 
days. During this time the teachers learn about the use of behavior 
modification, reading and math diagnosis, and the individualization 
of instruction. Also, time is spent in reading the D/P Guide. The 
teachers note all questions which arise while reading the guide and 
ask the D/P teacher to respond to them during the scheduled interview 
later on in the day. 

A session is also devoted to making teacher-made materials 
which have been specifically recommended for use with her student, 
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Along with the D/P Guide, the teacher receives a package of instructional 
material for use in the regular classroom with the returning* student . 

The Learning Specialists conduct follow-up activities when the 
students leave the DTC. They assist the regular classroom teacher in 
any way possible to help her implement the correct use of the D/P 
Guide and its recommendations. These activities have included ob- 
taining more materials for use in the classroom, restructuring the 
classroom for new groupings, setting up behavior modification pro- 
jects for target students, and demonstration teaching of a specified 
instructional strategy. Long-term follow-up is also conducted approx- 
imately at three month intervals. 

Evaluation Activities 

The DTC was funded under Title III, section 306 for a period 
of three years. The purpose of the Section 306 program is to test the 
appropriateness of specific solutions to critical national educational 
needs . Each project emphasizes its resources toward a limited 
number of students rather than try to serve directly all students 
who might benefit from the project. The Office of Education chose 
to focus on a limited number of projects across the nation to be 
funded at a sufficiently high level to provide for the development 
and delivery of a borad range of education related services to 
specific target populations. Fifteen percent of the funds allocated 
for Title III in June 1971 were for section 306 projects. Of the section 
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306 funds, fifteen percent were to be spent on projects for handicapped 
students, (A Manual for Project Applicants and Grantees, 1972) 

The guidelines for the operation of Section 306 projects state 
that an Evaluator will be employed for the project and that an Educa- 
tional Program Auditor will be employed to audit the functions of the 
evaluator. "The essential task of the auditor is that of reviewing the 
evaluation of a given project intent and anticipated performance out- 
comes." (Stenner & Webster, 1971, p. 21). The Auditor, then, is a 
reviewer of the activities specified by the Evaluator to insure that all 
reports are on target date and contain correct and appropriate infor- 
mation as previously specified. 

When the Office of Education specifies that performance object- 
ives are a requirement on which the evaluation design is to be based, 
and when the Evaluator notes that ". . .objectives are at best insuf- 
ficient as a foundation for evaluation" (Peck, 1974), interesting com- 
ments begin to appear in the Auditors report. As it now stands, the 
use of Information Based Evaluation is not widely accepted as total 
procedure for creating an evaluation design. 

The Evaluator for the first year of the project saw evaluation 
as an aid to project management. Evaluation was considered as an 
integral part of project management and frequently as the foundation 
for good program management. By considering evaluation as a dimen- 
sion of management, it was felt that the administration would acquire 



11 . 

an increased capacity to translate observations into action. This approach 
would provide the information required for guiding the future course of 
the project. (Evaluation Design, DTC, 1st Year, 1971). 

This approach to evaluation was definitely "future" oriented since 
it's purpose was to gain information to allow sufficient input into the 
second year's planning so that the criteria of the stated objectives 
would be at least reasonable. The objectives for the first year were 
essentially geared to OBSERVE the way things were operating at the 
initial stage of the project. 

While this approach to evaluation was established, the administra- 
tion of the school system was still concerned with "does the project do any 
good for kids". The answer would only be available during the second 
year after the students had spent a significant amount of time back in the 
regular classroom. 

The evaluation design for the second year began by viewing 
evaluation questions of interest and then writing objectives to determine 
if the questions could be answered. The use of traditional statistical 
approaches as well as descriptive statistics were included in the 
analysis of the objectives. Then an interesting thing happened. The 
Evaluator was promoted to a higher position within his firm, and a new 
Evaluator appeared on the scene. It was at this point that a more pure 
approach toward the use of Information Based Evaluation was instigated. 

The new Evaluator had successfully implemented this approach 
with some forty projects at both the state and local levels. He felt that 
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traditional methods of evaluation were too closely tied to the traditional 
research models and were too complicated and complex to allow the 
retrieval of information for decision-making. While the use of performance 
objectives was seen as meaningful to project management, their use was 
not viewed as the best method to insure complete information utility by 
decision-makers . 

It should be reiterated here that Information Based Evaluation 
might not be viewed in the same light by the Office of Education as it is 
viewed by the Evaluator consultant. In A Manual for Projcet Applicants 
and Grantees for Title III section 306, Draft of November 1972, the 
evaluation process is viewed totally via a performance objectives approach. 
The use of "Evaluation Design Summary Charts" is required. These 
charts display (a) objectives or performance to be measured, (b) measure- 
ment instruments and techniques, (c) data collection procedures, (d) data 
analysis techniques, and (e) data analysis presentation format. 

The project Evaluator felt that the traditional performance objec- 
tive evaluation approach was inadequate because of the following: 

1 . Basing evaluation on performance objectives restricts 

the focus of evaluation to intended outcomes, thus over- 
looking unintended outcomes which are potentially just 
as important. 

2. Performance objectives provide a very inflexable basis for 
evaluation in that they are seldom changed during the pro- 
gram year, and thus information needs (which are fluid) 
cannot be adequately addressed. 

3. Even if information on the attainment of all performance 
objectives is provided, important information is invariably 
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ignored because objectives are not developed with informa- 
tion needs in mind, but rather are developed as guideposts 
for program management. 

4. Objectives based evaluation often views each objective as a 
unique area of focus and thus important relationships are 
often overlooked. 

"In objectives based evaluation, the reference points are the program 
objectives. In IBE the reference points become the information users for 
the program and the information domains (needs). Capitalizing on these 
two reference points, a technique called domain analysis can be used to 
define and focus the direction of the evaluation" (Peck, 1973) 

The concept of Information Based Evaluation is not free of the use 
of objectives . Their use is justified when feedback on the objectives is 
important to information users. The questions asked by the users are of 
paramount importance in information based evaluation. The desired in- 
formation is then ranked in order of importance or priority as is the list 
of information users. Much of this work is done at the Design Conference. 

For the third year's evaluation of the DTC, such a design conference 
was held. In attendance were the Evaluator, who chaired the meeting, the 
superintendent, assistent superintendent for special services, assistant 
superintendent for curriculum, director of special education, and the 
assistantto the director of federal programs. Personnel from the DTC 
include the project director, psychologist, Language Specialist, Reading 
Specialist, and the evaluator's assistant. At this conference, many 
questions were raised. They were subsequently categorized into the 
following Information Domains: 
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1. 



Student Behavior Change 
Student Characteristics 



2. 



Teacher Behavioral Change 
Teacher Characteristics 



3. 



D/P Guide Effectiveness 
Prescription usage (etc.) 



4. Cost 

5. Attitudes toward the DTC: Teachers, Administra- 
tors (Principals), Staff, Support Staff, Parents, 
and Students. 



Specific Evaluation Questions by domains were: 



1 . How does the DTC effect student achievement? 

2. Do the students who participate in the DTC exhibit 
a significant educational change in academic and 
affective behavior? 

3. Are DTC techniques usable to quantify a child T s 
behavior in the classroom. 

4. How many students receive new assignments after 
their DTC work? 

5. What, if any, is the cause-effect relationship be- 
tween the D/P Guide and student behavioral change? 



6. What are the characteristics of teachers who suc- 
cessfully implement the D/P Guide? 

7. Is the D/P Guide transmitted from teacher to teacher? 

8. How does the DTC workshop training effect teacher 
behavioral change? 

9. Is there consistency in the application of the model 
among DTC staff? 



A. 



Student Behavioral Change 



B. 



Teacher Behavioral Change 
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C. Costs 

10. What is the per pupil cost of the DTC by pupils 
in attendance? 

1 1 . What is the per pupil cost of the DTC by tho;se 
affected? 

12. What is the cost of developing a D/P Guide? 

13. Can the utilization of the DTC as a resource 
reduce existing costs of instructing the mildly 
handicapped child? 

D. Attitude toward the DTC: 

14. What is the attitude of the following toward the DTC? 

teachers 
parents 
students 
staff 

support staff 
administ rators 

Once the Evaluator had the questions of interest, he asked the 
Design Conference personnel to list the users of the information in 
order of importance. The results were: 



1 . 


Board of Education 


2. 


Central Administration 


3. 


School Principals 


4. 


Auxiliary Services 


5. 


Teachers 


6. 


Public 


7. 


Project Staff at DTC 



With this information, decisions were made about the evaluation 
activities which were necessary to answer the questions Relative to 
Student Behavioral Change, the Evaluator decided that an experimental 
and control group study would be appropriate. By having the Design 
Conference prior to the beginning of the school year, time was avail- 
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able to arrange for the selection of the two groups with data collection 
beginning as school opened. On the other hand, much information in 
the Teacher Behavioral Change domain was decided to be collected 
by informal observational methods. 

The DTC has established a data collection procedure which 
contains over 125 elements of data per student enrolled. These data 
were loaned to the Evaluator for his use in answering the questions 
of interest. The questions were such that much information is required 
on a relatively small number of students because of the highly individual* 
ized nature of the project. Much of the data in the DTC data bank is 
regularly used by the project director. For instance, an attempt is 
made to group students of a similar age and grade level within each 
phase. Also, a variety of schools to be represented by students at the 
DTC is desired across all phases. By viewing the on-going DTC 
data bank information, adjustments can be made in the operational 
procedure for succeeding phases. These data were recently used to 
determine academic change for DTC students who had been returned 
to the regular class for a period of one year. Using data in the data 
bank as a pre-test, the Learning Specialists post -tested the appropriate 
students. This was a small sub-study within the confines of the overall 
evaluation design. 

The Evaluator, using his concept of Information Based Evalua- 
tion met frequently with the project director throughout the year in 
order to establish a time line of function. Usually this procedure is 
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one of working backwards from significant deadlines such as due dates 
for quarterly reports. At these meetings, discussion centered around 
when things had to happen in order to obtain the necessary data and 
information to meet deadlines. Similar meetings prior to the open- 
ing of school assisted in establishing a Time Line for use in managing 
the project throughout the year. 

Summary 

In summary, the concept of Information Based Evaluation is being 
used to evaluate a program for students with learning problems in a 
public school setting. The project is funded under Title III, Section 306 
and requires the use of an Evaluator and an Educational Program Auditor. 
While the Office of Education requires the use of performance objectives 
as a total framework of evaluation, Information Based Evaluation might 
be considered a broader approach to evaluation. To comply with the 
demands o^ the Office of Education, objectives were written for the 
project and presented in the required format. This approach can 
easily be incorporated within the overall concept of Information Based 
Evaluation. 

At the present time, the Evaluator must now begin the synthe- 
sizing of available data and must obtain required additional data. 
Since this is the last funded year, the report should be presented in 
such a way as to encourage the continuation of the project supported 
by funds from other sources. The per pupil cost of operating the 
program might be viewed as expensive by school administrators, yet 
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it should be borne in mind that the purpose of Section 306 was not to 
serve all students in need. Since the administration is typically 
interested in the analysis of hard data, such information should 
be included in the final report. 

When looking at the evaluation process for any program, it 
is recommended that one* concept of evaluation be adopted and main- 
tained through the life of the project insofar as possible. Additionally, 
the manager of the project should provide input to the consideration of 
the evaluation design. When the information users sense statistical 
information and performance objectives as useful, they should be 
built specifically into the design possibly as sub studies. With all 
this in mind, it seems that the use of Information Based Evaluation 
provides a procedure for establishing an evaluation design of merit. 



